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SECTION  I.  INTRODUCTION  AND  BACKGROUND 


1.1  INTRODUCTION 

Procedures  for  the  operation  and  maintenance  of  U.S.  Air  Force  (USAF)  equipment  and 
systems  are  contained  in  publications  called  Technical  Orders  (TOs).  A  formal  Technical  Order 
System  is  operated  to  guide  the  acquisition  and  preparation  of  new  TOs  for  specific  programs,  to 
control  the  distribution  of  TOs  to  organizations  having  a  need,  and  to  manage  the  TOs  throughout 
their  life  cycle.  In  order  to  acquire  TOs  for  new  programs,  a  Technical  Order  Management  Agency 
(TOMA)  is  established  within  the  organization  having  acquisition  responsibility,  (leaded  by  a  TO 
Manager. 

This  report  documents  a  study  conducted  by  BioTechnology ,  Inc.,  for  the  Air  Force  Human 
Resources  Laboratory  (  A FURL)  to  develop  guidelines  for  selection  of  format  options  for  proc  ure¬ 
ment  of  technical  data.  The  guide  that  lias  resulted  from  this  effort  has  been  prepared,  in  two 
volumes ■-  A  FI  I R  L-TR -80-50  and  AFIIRL-TR-80-51 ,  to  b<‘  a  basic  reference*  publication.  It  is 
intended  for  use*  by  all  individuals  involved  in  the*  acquisition  and  manage  me  nt  of  USAF  TOs,  with 
emphasis  on  the*  responsibilities  and  concerns  of  the*  TOM. As  in  those  commands  acquiring  TOs.  The* 
scope  of  tliei  guide*  includes  a  review  of  the*  USAF  TO  System,  extensive*  descriptions  of  procedural 
data  formats.  guidelines  for  determining  formats  to  satisfy  particular  user  needs,  and  guidance*  in 
various  aspects  of  TO  procurement,  de*\e*lopme*nt,  and  management. 

It  is  necessary  that  such  a  document  he*  responsive*  to  the*  changing  needs  e>f  the*  USAF  and 
the*  transfigurations  in  procedural  elata  techniques  that  are  continually  occurring.  It  has  therefore 
be*en  pre*pare*d  with  the*  intention  that  it  will  be*  periodically  revised  and  updated  to  prevent  it  from 
becoming  obsolete.  Subsequent  issue’s  can  incorporate*  new  information,  reflect  changing  policies 
and  regulation,  and  correct  deficiencies  and  e*rrors  which  may  have*  been  made*  during  the*  prepara- 
lion  of  e*arlier  versions. 


1.2  BACKGROUND 

t.2.1  BACKGROUND  AND  ISSUES.  Since  the*  military  draft  ended  and  the*  all -volunteer  force* 
(AVF)  was  starte*d.  there  lias  been  a  significant  change*  in  the  characteristics  of  personnel  in  the* 
military  service.  The*  arguments  over  the*  effects  of  this  change  are*  intense,  but  no  matter  how  these 
arguments  and  the*  statistics  are  reviewed,  there  are  military  personnel  performance*  problems.  All 
services  are*,  therefore,  looking  into  many  approaches  to  achieve  and  maintain  personnel  perfor¬ 
mance  within  the  climate  of  the*  AVF.  This  includes  maximizing  the*  use  of  job  performance  aids 
(JPAs). 

It  has  be*en  recognized  for  many  ye*ars  that  TOs  were  not  always  adequate.  Historically  ,  the 
content  and  format  of  TOs  has  been  dominated  by  engineering  data  and  printing  considcratiems 
and  only  modest  attention  to  user  ne*eds  was  evident.  In  order  to  compensate  for  limitations  in 
the  maintenance!  manual,  the  services  typically  conducted  extensive  training  programs.  Today, 
however,  it  is  frequently  asserted  that  the  performance  potential  required  of  the  TO  is  widening 
because  of  problems  such  as: 


•  Unit  training  costs  are  increasing,  while  budgets  are  decreasing. 

•  Personnel  costs  are  increasing. 

•  Equipment  maintenance  costs  are  increasing. 

•  Material  readiness  is  decreasing. 

•  Entry  level  skills  are  decreasing. 

•  Sy stem/ equipment  complexity  is  increasing. 

•  Personnel  turbulence— turnover— is  increasing. 

The  performance  achievement  essential  to  t  SAF  technical  jobs  has  traditionally  been  obtained 
through  selection  and  training,  and  occasionally  through  an  emphasis  on  human  engineering. 
Technical  manuals  have  served  more  as  a  backup.  However,  there  is  now  a  distinct  willingness  on 
the  part  of  Director  of  Defense  Research  and  Engineering  (DDR&E)  and  USAF  policymakers  to 
give  TOs  a  greater  role.  Perhaps  this  new  willingness  has  emerged  because  nearly  60  percent  of  total 
life  cycle  costs  is  attributable  to  personnel,  or  because  the  enormous  costs  of  training  are  being 
scrutinized  by  both  Congressional  and  Executive  Branch  watchdogs.  The  Department  of  Defense 
(DOD)-widc  costs  for  formal  training  now  exceed  $7  billion  per  year;  it  has  been  established  that 
on-the-job  training  (OJT)  and  other  training  cost  twice  that  amount.  Thus,  there  is  a  strong  interest 
in  deferring  the  heavy  investment  in  front-end  training  for  new  recruits  until  military  career 
interests  and  potential  have  been  established. 

However,  without  this  front-end  and  continued  training,  new  technicians  cannot  use  traditional 
TOs.  A  potential  solution  to  this  dilemma  is  to  utilize  innovative  TO  formats  that  can  obtain 
performance  from  relatively  untrained  technicians,  i.e.,  invest  in  the  “book”  rather  than  the 
“head.” 

1.2.2  JPA  RESEARCH.  Selecting  the  proper  format  or,  indeed,  even  knowing  what  formats 
exist  is  a  task  for  which  TO  Managers  have  not  been  prepared.  Extensive  documentation  of  formats 
exists,  the  result  of  over  20  years  of  JPA  research,  but  it  is  rarely  in  a  form  that  is  useful  to  a  TO 
Manager. 

The  Air  Force  Personnel  and  Training  Research  Center  (a  predecessor  of  AFI1RL)  was  in 
the  forefront  of  this  research,  experimenting  with  “behavioral  guides”  as  a  means  of  improving 
manpower  utilization  and  maintenance  effectiveness  in  the  mid-1950s.  Since  that  time,  all  branches 
of  the  military,  major  systems  contractors,  and  several  independent  research  and  development 
(R&D)  organizations  have  been  engaged  in  job  performance  aids  research.  In  one  sense  the  effort 
could  be  considered  overly  successful;  over  100  novel  formats  or  techniques  have  been  generated 
that  purport  to  replace,  supplement,  or  redesign  conventional  TOs.  These  techniques  include  not 
only  formats,  but  diagnostics,  delivery  systems,  media  utilization,  and  training  approaches. 

Unfortunately-but  not  surprisingly— not  all  of  these  prospective  innovations  have  been  useful. 
Some  have  only  been  minor  variations  from  conventional  techniques.  Some  have  been  so  esoteric 
as  to  be  impractical.  Some  have  been  tested  and  some  have  not,  but  few  have  been  implemented. 

The  lack  of  implementation  of  new  TO  formats  is  surprising,  considering  the  promise  shown 
by  the  research  and  development  efforts.  A  review  of  some  of  the  critical  research  and  a  relaxing  of 
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scientific  rigor  in  llic  interpretation  of  results  allows  sweeping  generalizations  to  lx*  made  with 
respeet  to  past  technical  data  R&l)  results,  as  is  presented  in  Table  l  - 1 . 


Table  1-1.— Reports  of  Data  Format  Tests  Reported 
by  Various  Researchers 


OATA  FORMAT  TESTS  (EXAMPLES): 

• 

FAVORABLE  RESULTS: 

-  SPARES  UTILIZATION . 

...REDUCED  30% 

-  MEAN  TIME  TO  REPAIR  (MTTR) . 

, . .  REDUCED  40% 

-  ON  LINE  MAINTENANCE  MAN-HOURS  . . . 

...REDUCED  33% 

-  FAULT  ISOLATION  TIME . 

...REDUCED  36% 

-  FORMAL  TRAINING  TIME . REDUCED  50%  TO  80% 

• 

UNFAVORABLE  RESULTS: 

-  LESS  ACCEPTABLE  TO  EXPERIENCED  TECHNICIANS 

-  SUPERVISORS  ARE  SUSPICIOUS 

-  FRONT-END  COSTS  ARE  INCREASED 

Kven  though  these  results  may  not  apply  to  all  systems  under  any  conditions,  the  ""evidence" 
from  prior  research  is  formidable  enough  to  justify  better  utilization  of  the  results.  This  is  not  to 
say  that  all  results  have*  been  favorable;  indeed,  there  have  been  problems  with  innovative  TOs. 
Nevertheless,  thorough  researchers  have  identified  the  problems  as  well  as  the  potential  benefits, 
and  the  total  spectrum  of  results  can  be  considered  in  selecting  formats  for  USAK  TOs. 

1.2.3  RESEARCH  PAYOFF.  When  considering. the  topic  of  technical  data  research,  it  is  reasonable 
to  speculate  on  the  projected  payoff  from  utilizing  some  of  this  past  research.  Organizations  within 
the  Air  Force,  Army,  Navy,  and  DDR&E  have  endorsed  the  statement  that  the  use  of  innovative 
formats  can  have  valuable  payoffs  such  as  those  indicated  in  Table  1-2.  For  example,  paragraph  1.6 
of  AFSCM  310-2  (Technical  Publications  Acquisition  Manual)  states: 

Ve w  methods  for  technical  publications.  In  the  early  stages  of  the  program,  the 
SPO  (System  Program  Officer]  should  consider  new  techniques  for  presentation  of 
maintenance  and  operations  information.  He  will  review  such  new  techniques  for 
application  to  his  program  as  a  supplement  or  replacement  of  existing  publications. 
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Table  1-2.— Projected  Payoffs 
and  Impacts  of  Innovative  Formats 


PROJECTED  PAYOFF:  USE  OF  INNOVATIVE  JPA*  CAN. . . 

•  DECREASE  TRAINING  COSTS 

•  INCREASE  PRODUCTIVITY  OF  ENTRY  LEVELS 

•  INCREASE  CAREER  FORCE  CANDIDATES 

•  DECREASE  SPARES  UTILIZATION 

•  DECREASE  MAINTENANCE  LABOR  FORCE 

•  DECREASE  MAINTENANCE  PERSONNEL  COSTS 

PROJECTED  IMPACT:  ACHIEVEMENT  OF  PAYOFF  REQUIRES.  .  . 

•  INCREASED  INITIAL  INVESTMENT 

•  IMPROVED  REQUIREMENTS  ANALYSIS 

•  INCREASED  DATA  ACCURACY 

•  MORE  T.O.  MANAGER  EXPERTISE 


No  new  research  is  needed  to  develop  new  formats.  The  problem  is  how  to  encourage  and 
facilitate  the  serious  consideration  of  available  format  options  by  the  TO  Manager  in  the  SPO. 
The  potential  benefits  will  never  be  realized  unless  the  institutional  users  (TO  Managers)  are  aware 
of  specific  formats,  their  benefits,  and  their  impact.  Furthermore,  the  TO  Managers  must  be  pre¬ 
pared  to  present  the  case  for  new  formats  in  terms  of  their  payoff  and  impact  on  other  USAF 
agencies  within  Air  Force  Logistics  Command  (AFLC),  Air  Force  Systems  Command  (AFSC),  the 
using  command,  and  Air  Training  Command  (ATC). 

1.3  STUDY  PURPOSE  AND  OBJECTIVES 

1.3.1  IDENTIFICATION  OF  NEEDS.  The  TO  Manager  for  a  System  Program  Office  (SPO), 
normally  in  the  AFSC,  has  a  difficult  and  demanding  job.  In  coordination  with  TO  representatives 
of  AFLC  and  the  major  command  involved,  the  TO  Manager  must  select  the  best  type  (format)  of 
TOs  available.  The  type  selected  should  meet  the  needs  of  the  specific  weapon  system  involved, 
as  well  as  the  needs  of  the  technicians  who  will  use  the  data.  Many  different  formats,  and  options 
within  each  format,  have  been  developed.  Most  formats  have  specific  strengths  and  weaknesses. 
Careful  selection  and  application  of  format  types  is  necessary  in  order  to  get  maximum  benefit 
from  the  positive  features  of  each  one.  However,  information  has  not  been  available  to  assist  the 
TO  Manager  in  determining  user  requirements,  or  in  selecting  the  best  available  format  to  fulfill 
the  requirements. 
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Research  results  from  AFHRL  and  from  other  Air  Force,  Army,  Navy,  and  industry  organiza 
tions  are  sometimes  not  readily  available  to  TO  Managers.  Often  the  information  is  not  in  a  usable 
form  when  it  does  become  available.  The  task  of  locating,  evaluating,  and  applying  the  T()-related 
research  results  of  the  various  Government  and  industrial  organizations  exceeds  the  time  available 
to  most  TO  Managers. 

Once  the  proper  format  has  been  selected,  another  difficult  aspect  of  the  TO  Manager’s  jot) 
begins— that  of  procuring  the  best  possible  technical  data  from  the  contractor.  Although  con 
siderable  research  has  been  done  on  improving  the  processes  for  developing  technical  data,  this 
information  is  not  readily  available  to  the  TO  Manager  in  a  concise,  usable  form.  A  need  exists 
for  a  guide  to  identify  the  available  format  options  and  suggest  guidelines  for  selecting  formats 
to  fit  specific  system  and  user  needs.  The  document  should  consolidate  the  appropriate  information 
in  one  publication,  in  a  format  and  style  readily  usable  by  USAF  TO  Managers.  It  should  include 
information  that  will  enable  better  management  of  a  TO  development  contract,  as  well  as  infor¬ 
mation  on  implementing  and  using  new  TOs  in  the  field. 

It  was  hypothesized  that  a  guide  with  the  above  characteristics  would  improve  the  quality 
of  TOs  by  making  research  and  experimental  information  available  to  the  TO  Manager,  along  with 
a  consolidated  description  of  the  requirements  of  the  TO  System. 

1.3.2  OBJECTIVES.  This  report  and  the  companion  guideline  publications  were  developed  under 
contract  to  AFHRL  in  response  to  the  identified  need.  The  study  which  led  to  their  publication 
was  fully  coordinated  with,  and  cosponsored  by,  AFLC. 

The  purpose  of  this  project,  conducted  by  BioTechnoiogy,  Inc.,  was  to  assemble  information 
that  would  be  useful  to  Air  Force  TO  Managers  and  other  personnel  concerned  with  acquiring 
technical  data  for  use  in  developing  TOs.  To  that  end,  three  distinct  objectives  were  established. 
These  were: 

a.  To  identify  TO  acquisition  requirements  and  problems,  along  with  existing  formats  and 
guidelines. 

b.  To  determine  the  specific  information,  requirements,  procedures,  and  guidelines  that  should 
be  made  available  to  the  TO  Manager. 

c.  To  develop  a  handbook  of  data  and  procedures  for  requirements  determination;  format 
selection;  and  acquisition,  development  and  implementation  of  Air  Force  TOs. 

The  approach  to  meeting  these  objectives  was,  first,  to  gather  information  relating  to  the  needs 
of  TO  Managers  by  interviewing  TO  professionals  from  both  the  Air  Force  and  private  industry. 
This  stage  of  the  work  also  included  a  review  of  available  formats,  specifications,  and  applicable 
literature.  The  second  phase  involved  analysis  of  these  data,  with  the  selection  of  candidate  formats 
and  guidelines  and  the  development  of  additional  guidelines.  Finally,  the  text  and  introductory 
presentation  materials  were  prepared. 

The  activities,  findings,  and  products  of  these  three  phases  are  briefly  described  in  the  following 
sections. 
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SECTION  2.  PHASE  1— INFORMATION  GATHERING 


The  initial  phase  of  tin  project  romprised  three  tasks,  all  involving  tin*  gathering  of  information: 

•  Task  1  Interview  TO  Managers 

•  Task  2  Idt'iit if\  available  formats  and  specifications 

•  Task  d  Review  literature  for  potential  material  to  lie  included. 

2.1  IMKKVIKWS  WITH  T  O.  MANAGERS 

2.1.1  PREPARATION.  This  initial  task  was  for  the  purpose  of  determining  from  representative 
\ir  Eoree  TO  Managers  what  the  real-world  problems  of  TO  acquisition  are,  ami  the  types  of 
materials  that  would  he  most  useful. 

Briefing  material"  were  prepared  whic  h  inc  luded  an  interview  protocol,  spec  ific1  topics  to  he 
addressed,  a  desc  ription  of  tin  proje  ct  bac  kground,  and  a  preliminary  outline*  of  the  guide*.  Other 
materials  included  a  preliminary  list  of  specifications  relating  to  teehnieal  manual  formats,  and 
sumtnurv  reports  of  two  field  tests  of  JP  As  conducted  by  AKIIK I.. 

Primarv  objec  tives  were  to  determine: 

•  Invents  of  the  Derelofttncnt  I'rorrss  The  documentation  should  emphasize  key  events  of  the1 
proc  ess,  e.g..  Te  c  hnic  al  ( )rder  Requirements  Conference  (TORC). 

•  Organizations  Inndrcd  in  the  I'roeess  Tin*  documentation  should  include*  organizations 
responsible  for  major  produc  ts  or  events,  e.g.,  Oklahoma  City  Air  Logistics  Center  (OG  ALC) 
as  the  t  S  \l  TO  distribution  center. 

•  hiftuls  and  I'rodnets  The  document  will  identify  inputs  which  precede,  and  products  which 
result  from,  each  event  of  tin*  TO  development  process,  r.g..  Maintenance*  Philosophy  is  the* 
produc  t  of  tin*  Integrated  Logistics  Plan  and  potentially  serve's  as  an  input  to  the  TORC. 

•  Sf,enfi<  ations  and  I'olieies  Where  possible,  each  event,  produc  t,  or  input  of  the  process  will 
be  keved  to  a  policy,  specification,  or  handbook  which  describes  its  purpose,  characteristics, 
or  use. 

2.1.2  INFORMATION  SOURCES.  Initial  planning  anticipated  discussions/ interviews  only  will) 
TO  Managers  in  various  A  ESC  SPOs.  As  a  result  of  early  teehnieal  discussions  with  AKHRL  project 
monitors,  it  was  agreed  that  representative  contractor  TO  Managers,  under  a  “matched-pair”  eon- 
crept.  should  be*  included.  This  approac  h  would  ensure  that  the  guide  would  reflect  an  appreciation 
of  the*  problems,  capabilities,  and  limitations  of  TO  contractors.  Subsequently  it  was  also 
determined  that  data  c  ollec  ted  would  he  incomplete,  and  probably  biased,  without  inputs  from  TO 
management  personnel  in  the*  \l*  LG.  As  a  result,  interviews  were  hold  with  TOMA  personnel 
in  three*  divisions  (  \eronautical  Systems  Division  ( ASD),  Elec  tronic  Systems  Division  (KSD),  and 
Spac  e*  and  Missile*  Systems  Division  (S  AMSD).  two  AELC  centers  (WR -ALC  and  OO  ALC),  Strate*gie 
\ir  Command  (S\C),  and  four  contractor  organizations  (Raytheon,  Autoneties,  Boeing,  and 
TRW.  Inc.).  Disc  ussant  organizations  arc*  identified  in  Table  2  I ,  together  with  the*  program  areas 


involved.  Ml  iiis(*u>>unl>  were  very  open  and  cooperative,  and  significant  quantitie>  of  refcrcnre 
material.'  from  these  ongoing  programs  were  obtained.  In  addition,  the  project  tram  part ieipatcd  in 
a  conference  of  \FST  \SI)  TO  management  personnel  where  many  project -relevant  problem.*  were 
diseus>ed.  and  in  a  Technical  Manual  Symposium  sponsored  by  thr  National  Security  Industrial 
\ssoriation  ( N SI  \ ). 


Table  2-1.— Information  Sources ,  TO  Manager  Interviews 


AFSC 

CONTRACTORS 

SYSTEM 

AFLC 

SAC 

E 

t/i 

i/ 

/°  <?/ 

ASD 

ESD 

SAMSD 

WR  ALC 

OO  ALC 

7t 

y 

PELS 

• 

r 

F  16  AIRCRAFT 

• 

PAVE  PAWS 

• 

• 

MINUTEMAN 

[ 

• 

• 

• 

• 

• 

• 

MX 

- i 

• 

• 

• 

• 

GENERAL 

• 

• 

• 

2.1.3  FINDINGS  OF  T.O.  MANAGER  INTERVIEWS.  \s  a  result  of  these  interviews  and  dis¬ 
cussions,  a  number  of  signific  ant  findings  emerged,  plus  sorin'  wliirb  we  re*  not  in  the  mainstream  of 
this  project.  Each  of  these  findings,  however,  had  an  influence  on  what  was  to  be  inc  luded  and  bow 
that  material  was  to  be  presented.  The  most  significant  findings  arc-  discussed  here. 

a.  Totvnlial  l  svrs:  The  population  of  potential  users  of  the  guide  is  much  broade  r,  and  more- 
varied,  than  had  been  anticipated.  An  Air  Force  TO  Manager  may  be  military  (officer  or 
noncommissioned  Officer  (NGO))  or  c  ivil  service;  in  some*  instances,  the  user  may  be  a 
support  or  technical  integration  contractor.  The  range  of  prior  experience  with  acquisition 
of  TOs  is  from  zero  to  more  than  20  years.  The*  guide  is  likely  to  have  greatest  utility  for 
the*  relatively  inexperienced  TO  Manager  assigned  to  relatively  small  programs,  and  may  also 
be*  of  value*  to  Data  Manage  rs,  who  often  initiate  the  TO  acquisition  process  before  a  TOM  A 
is  established. 

I).  I  svr  Needs:  Because  of  the  variations  among  potential  users,  and  also  because*  of  the* 
differences  in  orientation  and  structure  between  AFSC  divisions,  needs  are  not  constant 
between  sets  of  potential  users.  Knowledge  about  alternative  TO  formats  is  not  wide¬ 
spread  among  potential  users.  More*  specifically,  contractor  personnel  often  appear  to  be 
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more  aware  of  the  full  range  of  possible  formal  options  than  do  \F  personnel.  Partialis 
as  a  consequenee  of  unfamiliarity ,  AF  personnel  spend  little  time  in  considering  formal 
questions.  Their  main  preoeeupations  are  with  sueh  issues  as  their  role  responsibilities  and 
prerogatives  vis-a-vis  other  elements  within  the  Air  Foree  and  vis-a-vis  the  eontraetor.  (iiven 
the  explieit  seope  of  the  present  projeet.  the  most  relevant  needs  are  for  descriptions  of 
options,  the  sanetions  for  seleetion  in  offieial  1)01)  documentation.  and  elementary  guide¬ 
lines  on  the  relative  benefit  and  eost  aspeets  of  each  set  of  options,  (lost  relationship 
guidelines  will,  however,  be  the  most  difficult  to  provide  due  to  the  unavailability  of  reliable 
eost  data  on  acquisition  of  TOs,  conventional  or  otherwise. 

\spects  of  Statement  of  Work  fSOU')  Preparation:  Again,  practice  varies,  but  in  some 
instances  it  is  possible  for  the  SOW  to  be  prepared  prior  to  the  establishment  of  a  TOMA 
within  the  SPO.  The  Data  Management  or  Logistics  Management  staffs  frequently  undertake 
this  function  as  part  of  the  overall  procurement  planning  process.  In  other  instances,  tin* 
TOMA  prepares  the  SOW  or  can  be  involved  in  substantial  dealings  with  the  prime  con¬ 
tractor  in  the  form  of  interpretations  of  the  specifications.  Finally,  in  some  instances  the 
SOW  for  TO  procurement  is  delayed  until  well  after  the  prime  contract  lias  been  let.  In  such 
circumstances,  the  system  contract  agreement  contains  only  a  general  (standard)  reference 
to  TO  development. 

The  main  point  raised  by  interviewees  was  that  it  is  necessary  for  the  TO  Managers 
and  their  counterparts  in  Logistics  and  the  user  commands  to  have  a  good,  basic  knowledge 
of  the  overall  procurement  process  including  the  allowable  variations  among  divisions  in 
AFSC  in  order  to  understand  how  the  SOW  for  TO  procurement  fits  in.  Further,  it  must  be 
recognized  that  SOW'  preparation  is  an  iterative  process,  becoming  more  and  more  definitive 
as  system  hardware  development  progresses. 

Management  Procedures  During  TO  Development :  The  techniques  for  in-process  and 
pre-/post-publications  reviews,  validation,  and  verification  also  vary  across  systems.  A 
principal  factor  is  the  nature  of  the  relationship  between  the  SPO  and  the  prime  contractor. 
*n  some  cases,  particularly  those  in  which  the  agreements  have  been  in  foree  over  an  ex¬ 
tended  period,  the  decisionmaking  activity  can  be  collaborative.  While  there  are  instances, 
according  to  respondents,  of  prime  contractors  exploiting  the  ambiguities  of  TO  procure¬ 
ment,  the  general  impression  is  that  such  collaborative  arrangements  are  constructive  for 
the  basic  reason  that  the  contractor  personnel  tend  to  be  more  permanent  in  their  roles 
and  can  provide  forms  of  technical  expertise  that,  in  effect,  help  the  TOMA  deal  with 
the  intricacies  of  the  process. 

One  such  intricacy  reported  more  than  once  is  the  conflict  between  the  increasingly 
frequent  objective  of  100  percent  verification  and  the  availability  of  resources  to  conduct 
such  a  process.  The  ingredients  required  for  100  percent  verification  are  validated 
procedures,  actual  system  hardware,  representative  user  personnel,  and  participation 
(monitoring  of  tests)  by  appropriate  representatives  of  all  Air  Force  commands  that  are 
parties  of  interest  (i.e.,  Logistics,  Systems,  and  the  user).  The  “clout”  to  bring  all  of  these 
elements  to  bear  in  one  place  in  a  timely  manner  was  perceived  to  be  missing  by  many  of 
the  respondents. 
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e.  Implementation  of  TOs  in  I  he  Held:  \  number  of  particular  problems  an'  associated  with 
implementation  of  TOs.  but  there  appear  to  be  two  main  generic  issues.  First,  there  is  the 
general  ambiguity  about  [he  boundaries  of  responsibility  between  the  SPO  and  the  coiinter- 
pari  Logistics  Command  unit.  To  some  degree,  this  ambiguity  ran  extend  into  the  domain 
of  the  prerogative  of  the  user  command.  In  some  eases,  special  elements  of  the  user  com¬ 
mand  have  initiated  revisions  to  the  TOs  provided  for  what  are  reported  to  be  parochial 
reasons.  Conversely,  user  commands  feel  that,  too  often,  the  TOs  are  inadequate  or 
inappropriate  for  the  technicians  who  must  use  them. 

The  second  generic  factor  is  closely  analogous  in  that  it  is  reported  that  the  formal 
mechanisms  for  improv  ing  TOs  (e.g.,  via  the  A  FTC)  Form  22)  do  not  function  particularly 
well.  However,  no  respondent  offered  any  alternative  method  for  attaining  corrective 
responses  from  user  personnel. 

f.  Ne/ec/hig  format  Options :  Most  of  the  respondents  seemed  predisposed  against  inclusion 
of  non-standard  formats.  Reasons  were  quite  variable,  but  most  often  mentioned  were: 

•  "Formats”  are  too  costly  to  prepare  and/or  maintain; 

•  Conventional  TOs  are  just  as  effective  if  “properly  written”; 

•  Krrors  in  "format”  TOs  are  more  serious  because  the  technician  has  no  latitude  to 
work  around  problem  areas; 

•  New  formats  are  not  well  accepted  by  using  personnel; 

•  The  TOMA  has  no  latitude  to  specify  formats  not  encompassed  by  existing  AF 
specifications. 

g.  Overall  Summary:  Most  of  the  problems  articulated  by  respondents  pertain  to  matters 
that  are  formally  beyond  the  scope  of  the  present  project.  Amelioration  would  involve 
policy  modifications  at  the  Air  Staff  level  in  some  cases.  It  is  pertinent,  however,  to 
emphasize  such  factors  as: 

0  The  TOMA  is  susceptible  to  relatively  rapid  turnover  of  personnel ; 

•  There  exists  no  specific  job  description  for  TOM  A  personnel— it  is  not  widely  recognized 
as  a  career-relevant  assignment; 

•  There  are  some  general  but  no  specific  qualifications  involved  in  the  assignment  of  an 
individual  as  TOMA; 

•  The  problem  environment  for  the  TOMA  is  considerably  broader  than  that  of  selection 
of  alternative  TO  formats; 

•  Both  continuity  and  effectiveness  of  TO  procurement  require  explicit  considerations 
of  TO  development  as  early  in  the  development  cycle  as  possible; 

•  Authority  and  responsibility  for  TO  effectiveness  are  not  lodged  in  any  single  organi¬ 
zation  during  the  entire  life  cycle  of  the  system. 

These  are  not  the  only  conclusions  that  are  possible  to  assemble  from  the  exchange  with  the 
respondents  contacted.  However,  they  constitute  some  of  the  issues  that  should  help  make  the 
guide  as  relevant  as  possible  to  the  user  community. 
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2.2  IDENTIFY  AVAILABLK  FORMATS  AM)  SPFCIFICATIONS 

Flu*  second  (ask  of  the  in  formal  ion  gathering  phase  was  directed  toward  identify  ing  (hr 
'formats'*  which  were  potential  candidates  for  inclusion,  and  the  specifications  or  other  guideline?* 
to  support  their  preparation. 

2.2.1  FORMAT  IDFINTIFICATIOiN.  Potential  formats  were  identified  starting  with  prior  surveys 
of  JP\  techniques,  the  most  recent  of  which  carried  the  traditional  list  of  100  JP  \s  (Me/1  / ).  This 
list  was  expanded  and  supplemented  through  a  review  of  research  reports  and  through  interactions 
w  ith  sources  in  the  military  .  other  government  agencies,  and  industry . 

In  order  to  determine  which  techniques  were  potentially  valid  formats  for  the  guide,  each 
was  screened  and  categorized  as  to  its  principal  character  (c.g..  format  versus  media),  and  the 
existence  and  availability  of  descriptive  information,  te  st  data,  and  preparation  guidelines. 

Illustrated  in  Figure  2  1  is  an  excerpt  from  the  listing  prepared  during  this  task.  The  techniques 
were  listed  by  acronym  and  abbreviated  title;  where  none  existed,  an  abbreviation  was  arbitrarily 
assigned.  The  "bullets"  in  the  character  columns  indicate  the  principal  character  of  tin*  technique, 
and  an  *  V'  designates  significant,  but  not  principal,  characteristics. 

The  next  three  columns  show  the  findings  relevant  to  the  other  screening  criteria:  descriptive 
data,  test  data,  and  guidelines  for  preparation.  \nv  for  mat -based  technique  which  bad  all  three 
types  of  data  was  designated  a  potential  candidate  for  inclusion.  Those  techniques  that  were  not 
format-based,  or  that  were  definitely  lacking  in  one  or  more  types  of  data,  were  eliminated.  The 
initial  listing  of  possible  format  candidates  was  expanded  significantly  during  the  requirements 
analysis  phase  as  more  was  learned  about  the  format-based  techniques.  The  complete  listing  is 
incorporated  in  \  FI  III  I, -TR. -80-51. 
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Figure  2-1. -Sample  of  Format  Identification  Data 


10 


2.2.2  SPECIFICATIONS  AND  GUIDELINES  IDENTIFICATION.  The  sec  ond  portion  of  this  task 
dealt  uith  the  identification  of  spec ifiea lions  and  guidelines  documents.  Initially .  effort  focused  on 
references  in  the  format  literature  and  inclusion  in  the  LOLMP  Specification  List  Exhibit.  This  was 
expanded,  based  on  information  obtained  during  TOMA  interviews  and  references  in  Data  Item 
Descriptions  (1)1  Us).  Air  Force  Acquisition  Documents  (AFADs),  and  other  specifications.  A 
complete  analysis  of  the  listings  in  the  Department  of  Defense1  Index  of  Specifications  and 
Standards  (DOI)ISS)  was  accomplished  in  order  to  identify  specifications  used  by  the  other  services 
and  to  determine  current  versions  of  specifications  referenced  elsewhere. 

Since1,  as  a  result  of  the  TOMA  interviews,  TO  acquisition  and  management  guidelines  took 
on  increased  importance,  guidelines  documents  of  this  type  were1  also  included.  A  composite  listing 
of  all  potentially  useful  specifications  and  guidelines  documents  was  prepared,  identifying  the* 
doc  ument  and  its  most  recent  issue,  the*  services  using  the1  specification,  and  the  equipment  and 
manual  types  for  which  it  was  applicable. 

Illustrated  in  Figure  2  2  is  an  excerpt  from  the1  composite  listing  of  specifications  and  require¬ 
ments  documents.  The  completer  list  includes  Military  Handbooks,  Military  Specifications,  and 
Military  Standards,  as  well  as  manuals,  regulations,  and  similar  documents  applicable  to  specific 
branches  of  the  services.  In  addition,  documents  from  other  sources,  including  research  reports, 
arc  included  if  they  contain  useful  guidelines.  The  major  column,  headed  “Requirements."  will  be 
explained  under  the  discussion  of  specifications  analysis  in  Task  4.  The  complete  listing  is  incorpor¬ 
ated  in  AFIIRL  TR  80  51. 


Figure  2-2.- Excerpt  from  Specification  Identification  List 
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2.2.3  FORMAT/SPECIFICATION  IDENTIFICATION  FINDINGS.  The  major  findings  of  this 
task  were: 

•  There  are  more  acronyms  than  there  are  JPA  techniques;  some  techniques  an*  merely  renamed 
older  techniques;  and  some  are  not  techniques  at  all  (several  were  periodicals  directed  to 
maintenance  technicians). 

•  A  small  proportion  of  the  listings  are  format  based  techniques,  although  many  have  some  sig¬ 
nificant  format  characteristics.  Most  format-based  techniques  are  combinations  of  different 
formats.  Some  of  these  are  included  in  the  listing  in  their  own  right. 

•  Relatively  few  formats  are  supported  by  test  results  or  preparation  guidelines.  One  of  the  more 
interesting  revelations— since  tests  of  most  formats  compared  them  to  “conventional"  manuals— 
is  that  there  is  really  no  such  thing  as  a  conventional  technical  manual.  There  was  almost  as 
much  variation  among  examples  of  Conventional  Technical  Manuals  (CTMs)  as  among  format- 
based  J  P  As. 

•  There  appears  to  be  a  tremendous  amount  of  duplication  among  and  within  the  military  services 
in  creating  specifications.  Some  duplication  is  inevitable,  but  some  is  based  on  parochialism. 

•  Perhaps  more  serious  are  the  number  of  referenced  specifications  which  turn  out  to  be  super 
seded  or  canceled,  thereby  complicating  an  already  difficult  task.  A  related  finding  is  that  the 
AFADs— an  integral  part  of  the  TO  acquisition  process— are  seriously  out  of  date.  In  addition, 
the  project  team  was  unable  to  locate  any  official  documentation  which  specifically  explained 
how  and  when  to  use  an  AFAD. 

•  Finally,  there  were  entirely  too  many  specifications  and  related  publications  to  be  considered, 
forcing  the  team  to  concentrate  on  those  believed  to  be  most  significant. 

2.3  REVIEW  LITERATURE  FOR  POTENTIAL  HANDBOOK  MATERIAL 

Task  3  really  did  not  exist  as  an  independent  task,  but  was  the  vehicle  for  identifying,  locating, 
and  conducting  an  initial  review  of  publications  and  materials  to  support  other  tasks  in  both 
Phases  I  and  II.  Materials  were  extracted  from  documentation  prepared  in  current  and  past  acqui 
sition  programs,  research  reports  and  papers,  specifications  and  guidelines  documents,  and  Armed 
Forces  policy  and  regulatory  publications. 

A  comprehensive  bibliography  was  established  that  includes  more  than  130  research -type 
publications  used  directly  in  the  project;  this  bibliography  is  incorporated  in  AFHRL-TR-80-50 
and  AFHRL-TR-80  51. 


SECTION  3.  PHASE  II— REQUIREMENTS  ANALYSIS 


This  sec  ond  phase:  of  tin*  project  was  based  on  the  data  and  information  collected  in  Phase*  1. 
It  included  the  following  spe  cific  tasks: 

•  Task  4— Select  c  andidate*  formats  and  guidelines 

•  Task  5— Develop  additional  guidelines. 

3.1  SELECTION  OF  CANDIDATE  FORMATS  AND  GUIDELINES 

3.1.1  ANALYSIS  OF  CANDIDATE  FORMATS.  In  Task  4,  the  initial  step  was  to  analyze  detailed 
descriptions,  evaluations,  and  |e?,t  data  for  each  format -based  technique  re*sulting  from  Task  2.  A 
six-le*ve*l  hierarchy  of  formal  considerations  was  defined,  as  illustrated  in  Figure  3-1.  Because*  of  the* 
absenc  e*  of  consistent  terminology,  working  definitions  for  each  leve*l  w<*re  established. 


LEVEL 

NAME 

EXAMPLES 

1  _ 

_ JPA  SYSTLIW 

AF/FP.IPA 

o 

PRIMARY  FORMAT 

MIM 

3 

FORMAT 

JOBGUIDE 

4 

SUROROINATE  FORMAT 

MIF 

5  _ 

FORMAT  COMPONENT  _ 

NIIMRFQ.KFVFn  PHOTO 

6 

FORMAT  ELEMENT 

_  PHOTOGRAPH 

i - -  - - 1 

Figure  3-1  —Hierarchical  Relationships  Qf  JPA  Technologies 


a.  JP  1  System  (Level  I)— Includes  one  or  more  Primary  Formats  and/or  any  number  of 
Formats,  Subordinate  Formats,  and  Format  Components  and  Elements;  may  also  inelude 
“standard”  data,  me*dia,  delivery,  diagnostic,  and  training  characteristics  and  components. 
JPA  Systems  encompass  a  wide  range  of  data  applications;  for  ewample,  ope  ration  and 
maintenance!,  different  maintenance  levels,  or  all  aspects  of  maintenance  (removeVreplace, 
fault  isolation,  inspection,  lubrication,  etc.).  JPA  Systems  are  generally  well  organized 
approaches,  and  preparation  guideline's  are  contained  in  a  single*  specification  or  group  of 
specifications. 

b.  Primary  Formats  (Level  2)— May  include  any  numbt;r  of  Formats,  Subordinate  Formats,  and 
Format  Components  and  Format  Elements.  May  be*  use*d  with,  and  may  somctime*s  include, 
non-format  tee*hniqucs.  Primary  Formats  are*  ge*ne*rally  directed  toward  a  single  type  of 
activity  (e.g.,  troubleshooting)  and  may  or  may  ne>t  be*  part  of  a  JPA  System.  Primary 


Formats  arc  typically  complete  “packages"  requiring  little  (if  am )  supporting  or  supple¬ 
mental  data.  In  some  JI’A  Systems,  Primary  Formats  arc  structured  techniques  for 
organizing  operation  and  maintenance  information  according  to  the  type  of  activity.  In  such 
systems,  several  Primary  Formats  may  utilize  the  same  Format,  with  different  content. 

e.  formats  (/.ere/  .'lj — These  are  the  hasic  structures  that  characterize  a  particular  techniqiK . 
They  result  from  particular  combinations  of  a  limited  number  of  Format  Klements  into 
Format  Components,  then  into  Subordinate  Formats  or  directly  into  Formats.  In  hard 
copy  technical  manuals.  Formats  art'  often  complete  on  one  page  or  less.  The  Format  is 
repeated  as  often  as  needed,  with  the  technical  data  changing,  throughout  the  manual.  The 
“traditional"  manuals,  where  the  text  Format  continues  through  many  pages,  are  no  real 
exception;  the  Format  is  repeated  on  each  page  (or  more  frequently),  but  technical  content 
changes.  When  testing  of  JPAs  has  taken  place,  it  is  usually  at  the  Format  level,  although 
it  may  be  reported  at  (and  ascribed  to)  a  higher  level.  \  Format  presentation  must  In¬ 
sufficiently  comprehensive  to  initiate  and  complete  a  task. 

d.  Subordinate  Formats  (Level  f)-This  is  an  arbitrarily  established  category  between  the 
Format  and  Format  Component  levels.  It  serves  as  a  place  to  locate  special  treatments  of 
Formats  to  present  a  particular  kind  of  data.  In  addition,  a  Format  that  is  included  within 
another  (usually  more  complex)  Format  is  designated  a  Subordinate  Format  for  that 
application.  Presentation  approaches  that  are  used  only  to  support  one  or  more  Formats  are 
Subordinate  Formats.  Subordinate  Formats  are  not  present  in  some  Formats. 

e.  Format  Components  (Level  .-^-Combinations  of  Format  Elements  that  are  insufficiently 
complete  to  be  called  a  Subordinate  F  ormat  or  Format. 

f.  Format  Element  (Level  6) -The  basic  building  blocks  of  data  presentation;  e.g..  words, 
photographs,  numbers,  sy  mbols. 

Level  3,  the  Format  level,  was  established  as  the  baseline  for  analysis,  so  that  techniques  at 
levels  I.  2.  and  3  would  be  considered  for  the  handbook.  Next,  candidate  techniques  were  docu¬ 
mented  at  each  level,  and  correlation  tables  were  prepared.  In  both  documentation  forms,  extensive 
use  of  symbology  reminds  the  user  of  the  level  in  the  six -level  hierarchy. 

Characterization  of  candidate  Formats  was  accomplished  by  determining  the  format  option 
characteristics  (i.e.,  level  of  proeedurali'/.ation)  for  various  areas  of  application  (troubleshooting, 
remove/install,  etc.),  and  the  applicable  use  factors  (printed  materials;  information  format: sensory 
channel;  personnel  factors;  types  of  equipment). 

3.1.2  ANALYSIS  OF  SPECIFICATIONS  AND  GUIDELINES.  The  second  step  of  Task  4  was  an 
analysis  of  specifications  and  other  guidelines  documents  to  identify; 

a.  those  which  are  presently  referenced  in  the  AFADs; 

b.  those  containing  requirements  and  preparation  guidelines  for  format-based  techniques; 

c.  those  containing  requirements  and  guidelines  for  acquisition  of  technical  manual  data. 
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Tallies  were  prepared  idenlifv  ing  these  factors.  and  ineluded  in  the  handbook.  Figure  2-2. 
illustrated  earlier,  is  an  exeerpl  from  one  siieh  table.  The  requirements  eoliimn.  on  tin*  right-hand 
side  of  the  table,  ineliides  identifiration  of  the  Ivpes  of  requirements  imposed  on  the  eontraetor 
or  other  preparer,  and  notation  of  whether  the  speeifieation  includes  guidelines  for  data  acquisition 
and  for  the  preparation  of  J P  \  formats.  This  last  column  designates  those  specifications  which 
should  mateli  up  w it h  one  or  more  of  the  selec  ted  f  ormats.  IVimarv  Formats,  or  JIW  S\  stems. 


:u.:i  IM  K(;R\TI()N  AND  SKLKCTION  OF  FORMATS  AND  (UIDFUNFS  The  third  Step 
of  the  process  was  to  integrate,  or  mateli  up.  the  formal  speeifieations  with  the  format  based 
techniques.  f  igure  d-2  illustrates  an  exeerpl  showing  the  eorrelation  between  some  of  the  speei¬ 
fieations  identified  a>  having  format  requirements  and  the  format-based  techniques  that  were 
subjected  to  the  detailed  analysis.  The  complete  table  covering  the  Icchniqucs/spcrificalions 
selected  for  inclusion  is  incorporated  in  the  handbook. 


X  »  Basic  Requirement 
O  »  Specialized  Requirement 
>/-  Non-JPA  Requirement 


Figure  3-2. -Example  of  Requirements  and  Guidelines  Documents  for  the 
Preparation  of  Job  Performance  Aids  Format  Options 


The  final  step  in  Task  4  was  to  determine  which  format  based  techniques  should  he  recom¬ 
mended  for  inclusion.  The  ground  rules  established  for  this  selection  began  with  the  techniques 
that  were  already  determined  to  be  format- based  and  to  have  both  descriptive  data  and  test  data 
available. 

All  JPA  Systems,  Primary  Formats,  and  Formats  which  could  be  correlated  to  preparation 
guidelines  were  proposed  to  be  included.  In  some  cases,  the  Primary  Formats  and  Formats  were 
independent  of  JPA  Systems.  In  addition,  some  techniques  at  each  level  were  recommended  for 
inelusion  which  did  not  satisfy  all  of  the  selection  criteria  but  which  arc  closely  related  to  a  format- 
based  technique  that  did  meet  the  criteria. 

Based  on  these  recommendations,  a  selection  was  made  by  A  KURT  of  a  more  limited  set  of 
JPA  Systems,  Primary  Formats,  and  Formats,  as  follows: 

JOB  PERFORMANCE  AID  (JPA)  SYSTEMS 

•  AF/FPJPA  —  Air  Force  Fully  Proceduralized  Job  Performance  Aids 

•  JCiTOS  —  Job  Guide  Technical  Order  Sy  stem 

•  OMMS  -  Organizational  Maintenance  Manual  Set 

•  FOMM  —  Functionally  Oriented  Maintenance  Manuals 

•  SPA  —  Skill  Performance  Aids 

•  WORKPAC—  Work  Package  Concept 

PRIMARY  FORMATS  AND  FORMATS 

•  A  AT  —  Augmented  Action  Tree 

•  JOBGUIDE-  Job  Guides 

•  LTTA  —  Logic  Tree  Troubleshooting  Aids 

•  MDC  —  Maintenance  Dependency  Charts. 

3.2  DEVELOP  ADDITIONAL  GUIDELINES 

The  final  task  (Task  5)  in  Phase  II  consisted  of  seven  subtasks,  as  itemized  here: 

•  5.1  Determining  the  TO  Needs  of  the  User 

•  5.2  Selecting  the  Most  Appropriate  Format 

•  5.3  Suggested  SOW  Requirements 

•  5.4  Management  Procedures  for  the  TO  Development  Process 

•  5.5  Implementation  of  TOs  in  the  Field 

•  5.6  Obtaining  Maximum  Utilization  of  TOs 

•  5.7  Preparation  of  the  Bookplan. 

The  first  two  subtasks  relate  most  closely  to  the  format  issue.  The  next  four  arc  more  closely 
aligned  with  TO  acquisition,  regardless  of  format.  The  last  subtask  was  to  describe  the  expected 
content  of  the  handbook,  and  is  more  properly  discussed  in  Section  4. 
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3.2.1  FORMAT  SELECTION  DEPENDING  ON  USER  NEEDS.  This  activity  combined  the  goals  of 
subtasks  5.1  and  5.2  to  develop  an  integrated  set  of  guidelines,  based  on  available  methodologies, 
without  conducting  any  original  research. 

Finding  a  method  for  selecting  the  most  appropriate  formats  was  a  key  to  the  development 
of  a  useful  guide  for  TO  Managers.  Analysis  indicated  that  sophisticated  algorithms  would  be 
useless,  even  if  a  valid  algorithm  was  available.  Similarly,  a  “fully  proceduralixed”  aid  would  leave 
insufficient  decisionmaking  capability  to  the  TO  Manager. 

Consequently,  it  was  deemed  appropriate  to  prepare  a  general  procedural  guide  that  would 
assist  the  TO  Manager  in  determining  the  system  conditions  which  identify  certain  TO  needs,  and 
a  similar  guide  to  the  elimination  of  specific  format-based  techniques  which  fail  to  satisfy  the  TO 
needs.  This  process  has  been  termed  “Format  Option  Selection  Depending  on  Interaction  of  Con¬ 
ditions  (FOSI)IC).”  Although  this  method  differs  from  any  of  the  proposed  selection  techniques, 
it  is  based  in  part  on  the  method  proposed  by  T.J.  Post  (Re/.  2.3).  The  basic  steps  in  the  FOSDIG 
process  are  illustrated  in  Figure  3-3. 

The  guidelines  depend  heavily  on  the  preparation  and  use  of  a  topdown  breakdown  and  task 
identification  matrix,  preferably  prepared  by  the  contractor,  as  input  data.  The  full  description 
of  the  process  is  contained  in  AF1IRL-TR  80  50,  Section  6. 

The  approach  to  determining  the  TO  needs  of  the  user  is  based  on  identify ing  the  relationship 
between  the  conditions  of  a  system  and  the  system  personnel  who  would  be  using  the  TOs. 
Different  system  conditions,  su ch  as  the  time  expected  for  technicians  to  achieve  proficiency,  will 
determine  the  TO  format  requirements  that  will  best  compensate  for  that  system  condition,  or  have 
best  compatibility  with  it. 

As  shown  in  Figure  3  4,  some  condition  variables  will  require  a  specific  compensatory  format 
option;  other  variables  can  be  satisfied  by  many  of  the  TO  format  options.  Most  conditions  will  be 
matched  to  the  degree  of  proceduralization  required;  some  conditions  dictate  other  ty  pes  of  TO 
needs,  such  as  size,  organization,  and  automation. 

The  user  identifies,  for  each  significant  system  condition,  the  variable  which  applies  to  that 
maintenance  action  and,  by  extension,  the  format  option  indications.  Separate  guides  are  available 
for  troubleshooting  and  non-troubleshooting  maintenance  actions. 

Following  the  completion  of  the  guide  for  each  maintenance  action  on  each  item  of  equipment, 
a  composite  assessment  of  the  format  option  requirements  is  made.  At  this  stage,  the  user  is  able 
to  consolidate*  the  requirements  for  each  portion  of  a  system  for  which  maintenance  actions  are 
needed. 

The  TO  Manager  narrows  the  choices  of  format  options  and  makes  a  selection  of  specific 
formal-based  techniques  by  referring  to  a  series  of  correlation  tables  which  display  the  character¬ 
istics  found  in  various  JPA  Systems,  Primary  Formats,  and  Formats.  The  TO  Manager  then  makes 
a  final  selection  from  the  format-based  techniques  that  have  not  been  eliminated. 

3.2.2  ADDITIONAL  T.O.  ACQUISITION  GUIDELINES.  Regarding  the  SOW  subtask,  it  was 
determined  that  tin*  most  suitable  approach  was  to  provide  a  guide  to  the  items  that  should  be 
considered  in  the  preparation  of  a  SOW,  description  of  the  methods  available  for  placing  require¬ 
ments  on  contract,  and  identification  of  relevant  requirements  found  in  specifications. 


17 


Figure  3-3.-F0SDIC  Sequence  of  Activities 
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Variable 

Troubleshooting 

Nontroubleshooting 

Time  to 

Long  OJT 

Fully  proceduralized 

Proceduralized 

Proficiency 

Short  OJT 

Deductive 

! 

Descriptive 

Figure  3-4.— Example  of  System  Conditions  Influencing  User  Needs 


Tin*  diversity  of  programs,  organizations,  and  objectives  to  be*  satisfied  prohibits  the  develop 
inenl  of  a  "standard"  SOW.  The  specification  tables  previously  mentioned  provide  some  assistance 
to  the  TO  Manager  by  indicating  which  documents  eonlain  acquisition  requirements.  In  mam  eases 
these  specific  ations  contain  "ordering  data"  to  guide  the  TO  Manager  in  determining  the  specific 
requirements  to  be  placed  on  contract. 

The  Management  Procedures  subtask  focused  on  three  basic  areas:  ( 1 )  TO  Management 
Philosophies;  (2)  Fundamental  TO  Management  Tasks;  (3)  TO  Management  Tools.  In  ail  three 
are  as,  information  was  collected  from  a  broad  cross-section  of  specifications  and  other  guidelines 
documents  already  in  use*  by  the  services.  In  addition,  special  techniques  that  are*  appropriate*  to  the* 
procurement  of  ce  rtain  ly  pe*s  of  formats  we  re  included.  The*  approach  to  cove  rage*  of  [\\\>  topic 
was  te>  consolidate  the*  c*ollcctc*d  information,  making  e  very  attempt  te>  clarifv  ambiguous  are*as. 

Subtasks  5.3  and  5.6  were  concerned  with  identifying  and  analyzing  *%  jys  n>  yv  t^  h  TOs. 
e‘sp<*e  iall)  those*  involv  ing  unfamiliar  formats,  can  be*st  be*  introduced  to  if  id  personnel  and  with 
promoting  greate*r  utilization  e>f  the*  TOs  by  such  personnel.  This  topic  is  somewhat  troublesome 
as  far  as  guidance*  to  the*  TO  Manager  is  concerned,  largely  hceause  the  acquisition  manage  r  ha> 
ve*r)  limited  opportunity  to  interact  directly  with  the*  fie*!d  units  whe>  will  use*  the*  TOs.  In  addition, 
increased  utilization  of  TOs  is  actually  one*  e>f  the*  basic  reasons  for  all  e>f  the  preceding  tasks,  and 
of  the*  project  itself.  The*  premise  is  that  improved  formats  in  TOs  will  both  permit  and  pmrnotc 
increased  utilization. 

Based  on  this  premise.  twe>  approaches  we  re*  derived.  Kacli  are*a  in  the  handhetok  that  de  als  with 
topics  known  te>  affe*ct  the*  acceptance  and  utilization  of  TOs  (such  as  valuation  anel  verification 
to  ensure  elata  accuracy)  includes  specific  reminders  of  the*  re  lationship  of  that  topic  to  utilization. 
In  addition,  a  separate  discussion  is  included  in  an  appropriate  section  of  the*  guide.  The*  guidance* 
in  that  section  is  an  adaptation  e>f  the  approach  used  when  Job  Ouidc  Manuals  (JO Ms)  were* 
implemented  in  the*  (M4IA  pre>gram  (Kef.  i). 


SKCTION  4.  GUIDEBOOK  SCOPE  AM)  CONTENTS 
4.1  INTRODUCTION 

The  ultimate  objective  of  lliis  project  was  to  prepare  guidelines  that  t lit*  TO  Manager  would 
find  useful  in  all  aspects  of  TO  acquisition  and  management. 

Planning  for  guidebook  content  and  organization  actuall)  began  with  the  preparation  of  the 
preliminary  outline  in  Task  I.  During  arid  following  the  subsequent  informal  ion-gathering  and 
anal) sis  aetivities.  the  team  eonsidered  the  information  needed  hv  TO  Managers,  as  well  as  the 
information  available  for  inclusion.  Ollier  considerations  were  (he  arrangement  and  organization  of 
the  document  for  maximum  usefulness,  the  optimum  level  of  presentation,  features  to  preclude 
premature  obsolescence,  and  the  manner  of  production. 

Rased  on  those  considerations  and  the  data  developed  during  the  project,  it  was  concluded  that 
the  guidebook  must  be  both  a  practical  guide  and  a  reference  document.  It  must  deal  with  both  the 
normal  process  of  TO  development  and  acquisition  and,  separately,  with  the  considerations  for 
selection  of  format-based  techniques.  Only  that  information  which  is  verifiable  as  to  accuracy 
should  be  incorporated.  The  information  should  be  presented  in  a  manner  that  assists  the  TO 
Manager  with  decisionmaking,  but  does  not  limit  that  decision  capability.  And,  finally,  tin*  guide¬ 
book  should  be  produced  in  such  a  way  that  periodic  revisions  and  updates  will  be  practical. 

4.2  OVERVIEW 

The  guidebook  lias  been  prep  art’ d,  in  two  volumes,  to  be  a  basic  reference  publication.  It  is 
intended  for  use  by  all  individuals  involved  in  the  acquisition  and  management  of  Air  Force 
TOs.  with  emphasis  on  the  responsibilities  and  concerns  of  the  TOMAs  in  those  commands 
acqririug  TOs.  The  scope  includes  a  review  of  the  Air  Force  TO  System,  extensive  descriptions  of 
proc  edural  data  formats,  guidance  for  determining  formats  to  satisfy  partic  ular  user  needs,  and 
discussion  of  various  aspects  of  TO  procurement,  development,  and  management. 

4.3  GUIDEBOOK  SCOPE  AND  ORGANIZATION 

4.3.1  SCOPE .  The  TO  Managers  handbook  deals  with  two  principal  topics  of  significance  to  the* 
TO  Specialist  or  Manager: 

a.  The  Air  Force  TO  acquisition  c  ycle,  including  responsibilities,  methods,  and  data; 

b.  The  types  and  characteristics  of  procedural  data  formats  to  be  considered  for  use  in 
Air  Force  maintenance  manuals,  and  guidance  in  selection  of  the  most  appropriate* 
techniques. 

Other  topics  arc*  frequently  mentioned,  but  arc*  peripheral  to  these  two  central  issues. 
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4.3.2  ORGANIZATION.  Th<*  guidebook  for  TO  Managers  is  organized  into  two  volumes,  as  shown 
in  Table  4* I.  AFHRL-TR‘80-50  comprises  the  basic  text,  and  consists  of  14  sections.  AFIIRL-TR- 
8051  contains  10  sections  of  reference  data  and  supplementary  materials.  The  two  volumes  may 
be  used  together,  or  may  be  used  independently.  Some  data  an*  duplicated  in  the  two  volumes  to 
support  their  independent  use. 


Table  4- 1.  —Organization  and  Contents  of  the  TO  Managers  Guidebook 


AFHRL-TR-80-50  :  Technical  Order  Management  and  Acquisition 

Section 

General 

1 .  Overview 

Information 

2.  The  Air  Force  Technical  Order  System 

3.  Maintenance  Procedure  Format  Options— General 

4.  Format  Options:  Job  Performance  Aid  Systems 

Format 

5.  Format  Options:  Primary  Formats  and  Formats 

6.  Format  Option  Selection  Guidelines 

^  7.  Familiarization  of  Users  With  New  Formats 

8.  Technical  Order  Requirements  Determination 

TO 

9.  Technical  Order  Acquisition  Management 

Acquisition 

10.  Quality  Assurance  of  Technical  Orders 

-11.  Technical  Order  Publication,  Distribution,and  Revision 

Bibliography, 

'12.  Bibliography  of  Reference  Publications 

Glossary, 

13.  Acronyms  and  Abbreviations 

and  Index 

_14.  Topical  Index 

AFHRL-TR80-51 :  Technical  Order  Managers  Reference  Data 

Section 

r  1.  Glossary  of  Terms 

Glossaries 

2.  Abbreviations  and  Acronyms 

JPA 

3.  Job  Performance  Aid  Research  and  Development 

and  Format 

4.  Job  Performance  Aid  Concepts  and  Techniques 

Data 

5.  Format  Test  and  Evaluation  Data 

_  6.  Format  Option  Selection  Data 

Specifications 

7.  Technical  Manual  Requirements  and  Guidelines  Documents 

and 
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4.4  GUIDEBOOK  CONTENTS 


4.4.1  AFHRLTR-80-50— TECHNICAL  ORDER  MANAGEMENT  AND  ACQUISITION 

4.4. 1.1  General  Information:  Sections  1  and  2  contain  general  information  about  the  document 

and  the*  \ir  Force  TO  System. 

•  Section  I:  Overview  describes  the  purpose  and  objectives  of  the  handbook,  explains  its 
organization  and  contents,  and  recommends  approaches  to  effective  ways  to  use  it  as  an  aid  in 
solving  some  TO  acquisition  and  management  problems. 

•  Section  2:  The  \ir  force  TO  System  presents  a  concise  description  of  the  A  ETC)  System, 
consolidates  the  requirements  and  guidance  of  the  primary  TO  policy  and  regulatory  publica¬ 
tions,  and  describes  the  role  of  the  Air  Force  TO  Manager. 

4.4. 1.2  Technical  Order  Format  Considerations:  The  next  five  sections  of  this  volume  are  directed 

toward  explaining  the  ‘‘format  issue”  for  purposes  of  TO  acquisition  and  management. 

•  Section  3:  Maintenance  Procedure  Format  Options— (General  provides  a  brief  discussion  of  the 
background  of  procedural  data  format  research,  and  the  status  of  such  research  today.  Types  of 
formats  are  described  and  examples  are  provided  of  how  use  of  various  format  base  d  techniques 
can  help  solve  specific  procedural  data  problems.  A  general  orientation  to  the  format-descriptive 
information  in  Sections  4  and  5  is  also  provided,  as  well  as  an  overview  of  the  conventional  TO. 

•  Section  i:  Job  Performance  I  id  ( JPA )  Systems  presents  a  comprehensive  description  of  a 
selection  of  format-based  techniques  at  the  JPA  System  level.  This  includes,  for  each  technique, 
a  narrative  description  with  accompanying  illustrations,  a  hierarchical  breakdown,  format 
option  characterizations,  results  of  tests  and  experiments  involving  the  method,  and  identifi- 
c;  tion  and  description  of  specifications  or  other  preparation/acquisition  guidelines  which  arc 
available.  References  to  sources  of  additional  information  about  each  method  are  provided. 

•  Section  5:  Format  Options:  Primary  Formats  and  Formats  presents  a  comprehensive  descrip¬ 
tion  of  several  format-based  techniques  at  the  Primary  Format  and  Format  levels.  These 
techniques  are  either  independent  of  JPA  Systems  or  are  used  in  a  number  of  different  systems. 
Data  presented  arc  similar  to  those  in  Section  4. 

•  Section  6:  Format  Option  Selection  Guidelines  presents  guidance  for  determining  the  format 
options  which  will  best  satisfy  the  needs  of  the  user  in  a  particular  program  and  the  specific 
format-based  techniques  which  contain  those  format  options. 

•  Section  7:  Familiarization  of  Users  with  New  Formats  presents  a  discussion  of  the  need  for 
user  acceptance  of  TOs  if  they  are  to  be  effective  job  aids,  especially  if  a  new  or  unusual  format 
is  involved.  Methods  are  described  which  can  be  used  to  familiarize  using  personnel  at  all  levels 
with  the  features  of  the  TOs  being  provided.  Guidance  on  implementation  planning  and  conduct 
is  also  provided. 
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1.4. 1.3  Acquisition  ami  Management  of  TOs:  Tlii>  portion  dcaU  with  tin*  process  of  acquiring  and 

managing  \ir  l;orrr  TO>  regardless  of  the  partic  ular  loriiiat(s)  selected.  It  rove  rs  in  gre  ate  r  de  tail 

the-  \|  TO  System  ele*se  rihe  el  in  Se  e  lion  2. 

•  Sta  tion  H:  Tcthnital  Order  Requirements  bvtvrminntion  describes  liow  TO  requirements  arc 
identified.  the-  tv  p* >  of  data  whic  h  must  he*  considered.  tin*  pote  ntial  sources  of  sue  h  data,  and 
how  differences  in  the*  data  will  a  lie -el  tin-  types.  organization,  and  numbers  of  TOs  to  he* 
prov  ide-el. 

•  St'tfion  0:  TO  I tquisilitm  Management  provides  guidance-  in  tin-  preparation  of  work  state- 
ments  and  planning  documentation  and  if)  the*  interpretation  and  selection  of  spe-eifieations. 
Suggestions  for  eonlraetor  proposal  e  valuation  are  ineluded.  Emphasis  in  this  se  c  tion  is  on  the- 
need  to  effectively  document  the*  needs  of  a  program  and  evaluate  the*  contractor’s  response. 

•  Section  10:  Oualitv  \ssuranee  o/  TOs  describes  tin*  reviews  whic  h  should  occur  at  various 
points  iti  the-  development  of  TOs.  including  validation  and  verification.  A  composite  checklist 
of  review  criteria  is  included,  Guidance  is  also  provided  re-garding  coordination  with  and 
utilization  of  organizations  and  personnel  outside  of  the-  TO  acquisition  organization  to  provide 
re  view  assistance*,  and  regarding  planning  for  and  conduct  of  TO  verification.  Special  attention 
is  given  to  the*  two-ste  p  verification  proe*ess. 

•  Section  II:  TO  Tublic at  ion.  Distribution,  and  Revision  re-vie-ws  the*  requirements  for  printing, 
distribution,  and  revision  of  TOs.  Coverage  begins  with  the  initial  pJamtifig aetiv die*,  eontiruies 
through  actual  delivery  to  using  commands  anel  other  agencies,  and  concludes  with  utilizing 
inputs  from  field  activities  to  upgrade-  TO  quality  and  providing  feedback  to  TO  users  who  have* 
made  suggestions  and  recommendations. 

4.4.1 .4  Bibliography,  Glossary,  and  Index: 

•  Section  12:  tiiblioprafdiy  of  Reference  t*ubli<  ations  contains  full  bibliographic  citations, 
in  numerical  orde*r,  for  all  publications  referenced  elsewhere  in  the*  handbook,  as  we*ll  as  to 
other  re*ports  and  papers  that  could  assist  interested  TO  Managers  in  increasing  their  under¬ 
standing  of  format-based  te  chniques. 

•  Section  Li:  Acronyms  and  Abbreviations  is  a  comprehensive  listing  of  abbreviations  and 
acronyms,  with  the  full  title  supplied,  along  with  a  notation  as  to  whether  the*  abbreviation 
is  one  of  the*  many  J PA  techniques. 

•  Section  I  /:  Topical  Index  is  a  detailed  index  to  all  significant  topics  and  entries  in  AFHRb- 
TR-80  50. 


4.4.2  AFHRL-TR  80-51 -TECHNICAL  ORDER  MANAGERS  REFERENCE  DATA.  AFHRL- 
TR-80-51  contains  10  sections  of  reference  data  and  supplementary  information. 
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4.4.2. 1  Glossaries:  Materials  are  presented,  in  two  seetions,  that  will  aid  the  TO  Manager  in  under¬ 
standing  terminology.  Seetion  I  is  a  Glossary  of  Terms  wliieh  may  he  eneountered  during  TO 
acquisition  and  development,  and  includes  terms  used  by  the  Air  f  orce  and  other  military  services. 
Section  2  is  a  comprehensive  listing  of  abbreviations  and  acronyms,  with  the  full  title  supplied, 
along  with  a  notation  as  to  whether  the  abbreviation  is  one  of  the  many  JPA  techniques. 

4.4.2. 2  JPA  and  Format  Data:  Reference  information  on  formats  and  JPAs  is  presented  in  four 
sections.  Seetion  3  provides  a  discussion  of  the  background  of  format  ami  JPA  research,  including 
format  development,  test  and  evaluation,  and  selection  techniques.  Section  4  identifies  the  popula¬ 
tion  of  JPAs  which  have  been  proposed  for  use  at  various  times,  their  origins,  the  basic  character  of 
each,  their  development  status,  and  the  results  of  the  consideration  of  their  incorporation.  Section  5 
presents  summaries  and  excerpts  from  various  research  reports  concerning  the  testing  and  evaluation 
of  format-based  techniques.  To  the  extent  available,  the  discussion  covers  study  objectives, 
methods,  and  approaches,  and  both  quantitative  and  qualitative  results.  Section  6  contains  tables 
which  may  be  locally  reproduced  for  use  in  determining  optimum  maintenance  action  system 
conditions  and  in  selecting  format  options,  according  to  the  guidelines  in  AFHRh  TR  80-50, 
Section  6. 

4.4. 2.3  Specifications  and  Standards:  This  is  a  compilation  of  data  in  table  and  chart  form 
regarding  the  specifications,  standards,  and  other  requirements  and  guidelines  documents  that  are 
available  for  use  in  the  acquisition  and  preparation  of  technical  manuals.  Section  7  contains  a 
comprehensive  listing  of  requirements  and  guidelines  documents  that  are  available  for  use  in  the 
preparation  and  acquisition  of  technical  manuals  and  related  data.  Publications  from  both  military 
and  non  military  sources  are  included.  Section  8  provides  a  cross  reference  between  AFADs  used 
for  TO  procurement  and  the  specifications  and  standards  which  they  reference  or  incorporate. 
This  section  also  contains  tabular  results  of  an  evaluation  (Re/.  5)  of  Army,  Navy,  and  Air  Force 
Technical  Manual  Specifications. 

4.4.2.4  Bibliography  and  Index:  Section  9  contains  full  bibliographic  citations,  in  both  numerical 
and  alphabetical  order,  for  all  publications  referenced  elsewhere  in  the  handbook,  as  well  as  to 
other  reports  and  papers  that  could  assist  interested  TO  Managers  in  increasing  their  understanding 
of  format-based  techniques.  Section  10  is  an  index' to  all  major  topics  in  AFHRL-TR-80-51. 
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SECTION  5.  CONCLUSIONS  AND  RECOMMENDATIONS 

5.1  THE  GUIDELINES 

5.1.1  PATTERNS  OF  USE.  This  report  has  been  prepared  with  the  goal  of  serving  as  a  valuable 
aid  to  individuals  involved  in  all  aspec  ts  of  TO  acquisition,  development,  and  management.  Inevi¬ 
tably,  with  suc  h  a  broad  target  population,  the  value  of  the  document  will  varv  according  to  the 
past  experience,  levels  of  expertise*,  and  program  needs  of  each  individual.  TO  Managers  with  limited 
experience  should  derive*  the*  greatest  benefit.  For  these  individuals,  all  parts  of  AFHRL-TR  80-50 
can  serve*  as  a  text  for  their  profession  and  as  a  prac  tic  al  aid  in  overcoming  the  proble  ms  of  the  ir 
new  assignments.  Having  this  knowledge  will  not  be*  a  substitute  for  experience  as  a  TO  Specialist  or 
Manager,  but  it  will  help  avoid  some*  of  the*  pitfalls  of  TO  acquisition  and  management  that  the* 
experienced  individual  lias  alre  ady  faced. 

The  experienced  TO  Spe  c  ialist  will  very  like  ly  not  derive*  the*  same*  benefits.  Since*  this  user  has 
already  served  in  a  TO  program  capacity,  some  materials  will  be*  familiar.  On  tin*  other  hand,  the* 
discussions  and  descriptions  regarding  formats  may  be*  as  ne  w  as  te>  tin*  experienced  individual.  \s  a 
desk  reference,  the*  materials  in  AF1IRL  TR  80  50,  Se  ctions  4  and  5,  toge  the  r  with  the*  data  in 
AFII  RL-TR-80-51, should  be*  of  value*  to  both  the  experienced  and  inexperienced  individual. 

5.1.2  CONVERSION  TO  FORMAL  HANDBOOK.  When,  and  if.  the*  Technical  Reports  that 
resulted  from  this  project  are  converted  to  a  formal  Handbook,  the*  following  reeotnmeiulations 
should  be  implemented: 

•  The  Handbook  should  be  published  under  the  authority  of  both  the*  Systems  Command  and 
the*  Logistics  Command,  or  higher  authority.  We  believe*  this  is  ne*cessarv  so  that  it  is  recognized 
by  all  TO  Managers  as  representing  the*  interests  and  points  of  view  of  both  commands. 

•  The*  Handbook  should  be*  a  loose-leaf  document.  Revisions  and  updates  should  he*  planned  on 
at  least  an  annual  cycle.  This  will  permit  the*  Handbook  to  be*  ke*pt  reasonably  current,  reflecting 
changes  in  J PA  format  technology,  as  we*H  as  tracking  change  s  in  specifications.  Air  Force* 
policy,  regulations,  and  procedures.  This  approach  will  help  avoid  rapid  obsolescence*  of  the* 
Handbook,  increasing  the*  probability  of  its  continued  use*. 

•  Each  section  in  the  Handbook  should  be*  separated  by  labeled  tab  dividers,  te>  assist  the*  user 
in  locating  the*  particular  item  of  information  being  sought. 

•  Illustrations  should  be*  in  e*olor  if  the  subject  of  the*  illustration-such  as  a  particular  format 
would  normally  require*  the  use*  of  color.  Several  of  the*  JI#A  Syste*ms  re*comme*nde*d  use*  color 
coding  anel  color  shading,  and  they  cannot  be  ade*e|uately  illustrated  without  the  use*  of  color. 

5.2  OTHER  FACTORS 

In  the  process  of  e*onducting  the  analysis  and  preparing  the  guide,  several  other  ite*ms  of 
significance  were  identified  that  are  bejyond  the  scope  of  the  current  project  for  resolution. 
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a.  Tin*  present  ambiguity  regarding  responsibilities  for  various  aspects  of  TO  acquisition 
increases  the  difficulties  of  an  already  diffieull  task:  inappropriate  and/or  poor  quality 
I  t  )s  often  result. 

b.  TO  Managers  are  or  should  be  professionals.  A  career  field  should  be  established,  and  a 
mandator)  (raining  course  implemented.  The  currently  planned  training  course,  which 
lias  been  promoted  by  serious  TO  Specialists  for  almosl  10  years,  should  he  implemented 
as  soon  as  possible. 

c.  The  VITO-22  TO  Improvement  Reporting  System  enjoys  the  unique  distinction  of 
practically  no  differences  of  opinion.  It  was  consistently  identified  as  re  quiring  improve¬ 
ment,  particularly  in  tin*  level  of  responsiveness  to  change  requests  and  in  providing 
feedback  to  the  requesting  organization  and  individual.  Introduction  of  the  AFTO-27 
Publication  Change  Request  (FOR)  and  the  two-step  verification  process  may  eliminate  the 
sources  of  some  of  the  complaints,  blit  the  system  itself  appears  in  need  of  streamlining. 

d.  As  mentioned  earlier,  TO-related  AFADs  are  seriously  out  of  date.  The  revision  cycle 
should  he  (  hanged  to  keep  these  key  documents  from  being  obsolete. 

e.  Finally,  all  activities  related  to  the  acquisition  and  maintenance  of  TOs  should  be  clearly 
identifiable  on  a  cost  basis.  At  present,  no  one  really  knows  what  tin*  TOs  for  a  particular 
program  cost;  life-cycle  cost  comparisons  of  various  formats  are  not  possible. 

The  authors  believe  that  the  five  recommendations  listed  here  are  deserving  of  the  active  support 
of  AFLOaml  A FSC,  particularly  AKIIRL. 
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